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DETAILED ACTION 

1. This communication is in response to Amendment filed 01/04/2006, claims 1-24 and 53 have 
been canceled and claims 25-52 and 54-55 remain pending and have been examined. 

Claim Rejection under USC 101 

2. Claims 25-52 and 54-55 are rejected under 35 USC §101 which reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, 
or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of 
this title. 

3. Computer implemented method claims 25-52, computer-readable medium claim 54 and 
computing device claim 55 are rejected under 35 U.S.C. §101 because the claimed invention is directed to 
non-statutory subject matter. 

In this case, computer-related inventions whether descriptive or functionally descriptive material 
are non-statutory categories when claimed as descriptive material per se (see Warmerdam, 33 F.3d at 
1360 USPQ2d at 1759), falling under the "process" category (i.e. inventions at that consist of a series of 
steps or acts to be performed). See 35 U.S.C. 100(b) ("The term process means, art, or method, and 
includes a new of a known process, machine, manufacture, composition of matter or material"). 
Functional descriptive material: "data structures" representing descriptive material per se or computer 
program representing computer listing per se when embodied in a computer-readable media are still not 
statutory because they are not capable of causing functional change in the computer (see MPEP 2106). 

The method claims 25-52 do not seem to be described as being implemented in any tangible 
and/or limited to any tangible embodiment(s) (e.g. hardware components) in view of Applicant's 
disclosure. As such, the claim is not limited to statutory subject matter and is therefore non-statutory. 

The current guidelines set forth on invalidate examiner's previous recommendation. Further brief 
examination in this regard is provided below. Please see "Interim Guidelines for Examination of 
Patent Applications for Patent Subject Matter Eligibility (signed 26 Oct 2005; Published in the 
OG 22 Nov 2005 http://vmw.uspto.gov/web/offices/com/sol/og/2005/week47/patgupa.htm). 
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Claim Rejection under 35 USC §103 

4. Quotation of 35 U.S.C. § 103(a) which forms the basis for all obviousness rejections set forth in 
this Office action may be found in previous office action. 

5. Claims 25-52 and 54-55 are rejected under 35 U.S.C. 103(a) as being unpatentable (US 
6,564,263) Bergman et. al. (Bergman hereafter) in view of International Organization of Standardization, 
Organization International Normalization, ISO/IEC JTC1/SC29/WG11, Coding of Moving Picture and 
Associated Audio, MPEG 98 (MPEG-7 Evaluation Process Document), Atlantic City, October 1998 p. 1- 
75 (ISO hereafter) 

Regarding claim 25, Bergman teaches 

identifying multimedia type or modalities in multimedia content (col 3/lines 21-36, description 
data type, col 12/lines 23-64, select identified type: col 19/lines 41-49); 

capture (extract) multimedia object features, attributes or modalities from multimedia objects to 
form multimedia object descriptions and entity relation descriptions from the multimedia object (col 
3/lines 37-51, col 6/lines 39-43, extract: col 19/lines 54-58); 

generating entity relationships descriptions and multimedia object descriptions from a multimedia 
category, type or modality (develop: col 14/lines 44-48, create: col 12/lines 43-50, generate: col 19/lines 
50-53); 

generating "non-hierarchical entity relation graph descriptions", i.e. general relationships 
descriptors among multimedia objects (spatial relationship of object within/contained, col 17/lines 10-19, 
object within another object, col 17/lines 49-52, including objects with spatial, temporal relationships and 
thus allows the description of these relationships between presentation object col 3/lines 37-51, see Fig. 5- 
6 and using a graphic block-based representation to describe spatial and temporal relationships between 
the objects col 5/lines 64-col 6/line 14) and 

aggregating, incorporating or combining, i.e. integrating from the multimedia object descriptions 
(col 12/lines 17-26), entity relation graph descriptions (col 3/lines 27-36, 46-51, 59-62, Figs. 3-7, col 
4/lines 20-30, integrated multimedia descriptions and entity relationships descriptions: col 6/lines 57-64, 
col 8/lines 42-67, entity-relationships, col 9/lines 55-col 10/line 10) to generate an object ("description 
record"); 

wherein a composite object supports embedding of multimedia contents (col 7/lines 2-6, 
embedded hyper-linking, col 15/lines 4-17, spatial relationship of object within/contained, col 17/lines 
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10-19, object within another object, col 17/lines 49-52); however does not explicitly teach where 
descriptions are associated with communication between the objects. 

ISO discloses the graphical representation of the proposed description schemes follow the 
Universal Modeling Language (UML) notations for descriptors (D) and description schemes (DS) used 
address the diversity of approaches to modeling object orientated systems combining several modeling 
notations, including 

sequence diagrams which shows dynamic collaboration between a number of objects in particular 
the messages sent between objects and their temporal interrelationships (p. 71-72), Diagram (A1.2) shows 
an non-hierarchical entity relation graph, p. 75) 

It would have been obvious to one ordinary skilled in the art at the time the invention was made 
to include the from the ISO disclosure in Bergman's system because in doing so the generation of his 
entity relation description based on the multimedia object descriptions for multiple multimedia content 
types may also address the diversity of approaches to modeling object orientated systems combining 
several modeling notations, as suggested by ISO. 

Regarding claim 26, multimedia object pyramid (hierarchy) descriptions for one of the multimedia types 
(Figs. 3-4 & 8, hierarchical model of multimedia object descriptions based on content type; col 8/lines 12- 
16, 20-67). 

Regarding claim 27, the multimedia types includes image (Bergman: col 19/lines 41-49). 

Regarding claim 28, separating (segmenting) each multimedia content into descriptor defining portions 
(segments) including content from one of the multimedia type for the multimedia content (Bergman: col 
8/lines 55-58); and 

generating one feature representing a feature for one of the portions by feature extraction and 
annotation (Bergman: feature description: col 8/lines 42-46, annotation: col 8/lines 55-58, extraction col 
3/lines 37-51); 

wherein the generated multimedia object descriptions comprises one feature description for one 
segment (Bergman; col 8/lines 20-41). 

Regarding claim 29, wherein the segments are selected from the group consisting of local segments and 
global segments (Bergman: col 8/lines 55-58). 
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Regarding claim 30, feature description from the group consisting of media, semantic and temporal 
features (Bergman: col 8/lines 42-67). 

Regarding claim 31, a feature description selected from the group consisting of data location, scalable 
representation and modality trans-coding (Bergman: col 9/lines 25-37). 

Regarding claim 32, wherein the semantic features are further defined by one feature description selected 
from the group consisting who (Bergman: col 8/lines 47-50). 

Regarding claim 33, temporal features are further defined by one feature description consisting of 
duration (Bergman: objects duration, col 16/lines 16-34). 

Regarding claim 34, the applied prior art further teaches generating media object descriptions from the 
multimedia segment for one of the multimedia types by media object extraction processing (Bergman: 
capture (extract) multimedia object features, col 3/lines 37-51); 

generating media object hierarchy descriptions from the generated media object descriptions by 
object hierarchy construction and extraction processing (Bergman: col 14/lines 44-48;); and 

generating media entity relation graph descriptions from the generated media object descriptions 
by entity relation graph generation processing (Bergman: col 3/lines 27-36, 46-51, 59-62, Figs. 3-7, col 
4/lines 20-30, col 6/lines 57-64, col 8/lines 42-67, entity-relationships, col 9/lines 55-col 10/line 10). 

Regarding claim 35, segmenting the content of each multimedia type in the multimedia object into 
segments within the multimedia object by media segmentation processing (Bergman: col 3/lines 21-36, 
col 12/lines 23-64); 

generating one feature description for one of the segments by feature extraction and annotation 
(Bergman: feature description: col 8/lines 42-46, annotation: col 8/lines 55-58, extraction col 3/lines 37- 
51); 

wherein the generated media object descriptions comprise the feature description for one of the 
segments (Bergman; col 8/lines 20-41). 

Regarding claim 36, substantially the same as claim 30, same rationale of rejection is applicable. 
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Regarding claims 37-40, wherein generating media object pyramid (hierarchy) descriptions generates 
terminal/composite objects define as multimedia content pyramid (hierarchy) descriptions of the media 
object descriptions (Bergman: col 3/lines 27-36) based on relationships of media objects represented by 
the media object descriptions (Bergman: col 3/lines 41-51), and wherein the relationships consisting of 
media feature relationships, semantic feature relationships, temporal feature relationships, and spatial 
feature relationships (Bergman: col 6/line 1-14, 57-67, col 7/lines 10-14, 20-25). 

Regarding claims 41 and 46, wherein generating media entity relation graph descriptions (Bergman: col 
9/lines 55-col 10/line 10) generates entity relations graph descriptions of the media object descriptions 
based on entity-relationships and dependency-entity relationships of media objects represented by the 
media object descriptions (Bergman: col 14/lines 45-48, col 1 9/lines 50-53), 

wherein the relationships are selected from the group consisting of media feature relationships, 
semantic feature relationships, temporal feature relationships and spatial feature relationships (Bergman: 
col 6/lines 1-14, 57-67, col 7/lines 10-14, 20-25). 

Regarding claims 42-45, wherein generating multimedia object pyramid (hierarchy) descriptions 
generates multimedia object hierarchy descriptions of the multimedia object descriptions based on media 
content relationships of multimedia terminal/composite objects represented by the multimedia object 
descriptions (Bergman: col 11/line 27-33, inter-feature relationships: col 1 9/lines 50-53, inter-object 
relationships, col 15/lines 10-15); based on temporal, spatial feature relationships of multimedia objects 
(Bergman: col 15/lines 10-15); and semantic feature relationships of multimedia object (Bergman: col 
6/lines 58-64). 

Regarding claim 47, receiving and transcoding (encoding) the multimedia object descriptions into 
encoded description information (Bergman: col 7/lines 20-24), and storing the encoded description 
information as one (description) record (Bergman: col 8/lines 55-58). 

Regarding claim 48, this claim combines limitation(s) substantially the same as claims 25, and 47, same 
rationale of rejection is applicable 

Regarding claims 49 & 50, the encoding comprises binary encoding (Bergman: col 1 3/lines 31-33). 
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Regarding claims 51 & 52, the encoding comprises the extensible Markup Language (XML) encoding 
(Bergman: col 14/lines 4-18). 

Claim 53 (canceled) 

Regarding claim 54 this claim comprises the computer-readable medium storing instructions for 
performing substantially the same steps/acts discussed in the computer-implemented method on claim 25, 
same rationale of rejection is applicable. 

Regarding claim 55 this claim comprises the computing device including the modules configured to 
perform substantially the same steps/acts discussed in the computer-implemented method on claim 25, 
same rationale of rejection is applicable. 

Response to Arguments 

6. Applicant's arguments with respect to at least claim 25 have been considered but are moot in view 
of the new ground(s) of rejection. 

7. Regarding claim rejection under 35 USC 101 because the claimed invention is directed to non- 
statutory subject matter. In this case, according to these provided guidelines there has been some 
clarification as to how to interpreted the statutes in view of recent court decisions. 

"d. Machine Implemented Test: Whether a claim recites a machine implemented process is not 
determinative of whether that process claim is statutory. See, e.g., Benson, 409 U.S. 63, 175 USPQ 673 
(finding machine-implemented method of converting binary-coded decimal numbers into pure binary 
numbers unpatentable). However, using the machine implemented test, the claim would be found to be 
statutory. The Federal Circuit also recognizes that the fact that a nonstatutory method is carried out on a 
programmed computer does not make the process claim statutory. Grams, 888 F.2d at 841, 12 USPQ2d 
at 1829 (claim 16 ruled nonstatutory even though it was a computer-implemented process). 

In addition, the Federal Circuit has recently noted that a "structural inquiry is unnecessary" 
when determining whether a process claim is eligible for patent protection. AT&T, 172 F.3d at 1359, 50 
USPQ2d at 1452. Thus, a finding that a claim fails to recite a computer-implemented process is not 
determinative in whether that claim passes muster under Sec. 101. Therefore, USPTO personnel should 
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no longer rely on the machine implemented test to determine whether a claimed invention is directed to 
statutory subject matter." 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is 
reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS from 
the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing 
date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH 
shortened statutory period, then the shortened statutory period will expire on the date the advisory action 
is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later than SIX 
MONTHS from the date of this final action. 



10. The following prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. Copies of Non-Patent Literature documents cited will be provided as set forth in MPEP§ 
707.05(a): 

A. Geometry Coding and VRML, Taubin, G., et. al., IEEE, vol. 86, No. 6, June 1998, IEEE 0018- 
9219/98, P. 1228-1243. 

Taubin, G., et. al. discloses a scene graph description associated with communication between 
messages exchanged between multimedia objects called events over a connection between them called 
routes. 

Any inquiry concerning this communication or earlier communications from the examiner should 
be directed to Prieto, B. whose telephone number is (571) 272-3902. The Examiner can normally be 
reached on Monday-Friday from 6:00 to 3:30 p.m. If attempts to reach the examiner by telephone are 
unsuccessful, the Examiner's Supervisor, Andrew T. Caldwell can be reached at (571) 272-3868. Any 
inquiry of a general nature or relating to the status of this application or proceeding should be directed to 
the receptionist whose telephone number is (703) 305-3800/4700. 
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Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system, status information for published application may be obtained from 
either Private or Public PAIR, for unpublished application Private PAIR only (see http://pair- 
direct.uspto.gov or the Electronic Business Center at 866-217-9197 (toll-free). 

Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
P.O. Box 1450 
Alexandria, VA 22313-1450 

Hand carried or delivered to: 

Customer Service Window located at the Randolph Bldg. 
401 Dulany St. 
Alexandria, VA 22314 

Faxed to the Central Fax Office: 

(571) 273-8300 (New Central Fax No.) 

Or Telephone: 

(571) 272-2100 for TC 2100 Customer Service Office. 



